This section is intended to provide a background or context to the invention that is recited in the claims. The description herein may include concepts that could be pursued, but are not necessarily ones that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, what is described in this section is not prior art to the description and claims in this application and is not admitted to be prior art by inclusion in this section.
A video codec typically comprises an encoder that transforms input video into a compressed representation suited for storage/transmission and a decoder that can uncompress the compressed video representation back into a viewable form. Typically, the encoder discards some information in the original video sequence in order to represent the video in a more compact form, i.e., at a lower bitrate.
Typical hybrid video codecs, for example ITU-T H.263 and H.264 codecs, encode the video information in two phases. In the first phase, pixel values in a certain picture area (or “block”) are predicted. For example, motion compensation mechanisms may be used to find and indicate an area in one of the previously coded video frames that corresponds closely to the block being coded. As another example, spatial mechanisms may use the pixel values around the block to be coded in a specified manner. If a block is predicted by motion compensation mechanisms, it is referred to as an inter-coded block or an inter-block. If a block is predicted by spatial compensation mechanisms, it is referred to as an intra-coded block or an intra-block. Second the prediction error, i.e. the difference between the predicted block of pixels and the original block of pixels, is coded. This is typically accomplished by transforming the difference in pixel values using a specified transform such as a Discreet Cosine Transform (DCT) or a variant thereof, quantizing the coefficients and entropy coding the quantized coefficients. By varying the fidelity of the quantization process, the encoder can control the balance between the accuracy of the pixel representation (i.e., the picture quality) and the size of the resulting coded video representation (i.e., the file size or transmission bitrate).
The decoder reconstructs the output video by applying prediction mechanisms that are similar to those used by the encoder to form a predicted representation of the pixel blocks (using the motion or spatial information created by the encoder and stored in the compressed representation) and prediction error decoding (the inverse operation of the prediction error coding, recovering the quantized prediction error signal in the spatial pixel domain). After applying prediction and prediction error decoding mechanisms, the decoder sums up the prediction and prediction error signals (pixel values) to form the output video frame. The decoder (and encoder) can also apply additional filtering mechanisms to improve the quality of the output video before passing it for display and/or storing it as prediction reference for the forthcoming frames in the video sequence.
The coding performance of intra-coded blocks depends heavily on the availability of the pixels around the block for prediction. However, the neighboring pixels may be unavailable for prediction for various reasons. For example, such neighboring pixels could be unavailable if (1) the coded block is on a horizontal or vertical boundary of the picture; (2) neighboring blocks are inter-coded, and intra-prediction from inter-coded blocks is restricted in order to reduce spatial propagation of a potential error in transmission; (3) horizontal prediction is restricted for easy pipelining operation, and pixels that are horizontally adjacent to the coded block are unavailable; and (4) neighboring blocks belong to different slices, and intra-prediction between slices is restricted so that each slice is an independently decodable entity. If the neighboring pixels are not available for prediction, then the coding performance for the block is degraded significantly. This problem becomes even more severe if a larger size transform (and larger sized blocks) is used for coding the blocks within the macroblock, as a larger area of the macroblock would be coded without a good prediction compared to the smaller transform case. The problem is depicted in FIGS. 1(a) and 1(b). FIG. 1(a) shows the effect of unavailable pixels on intra-prediction based upon 4×4 blocks. In contrast, FIG. 1(b) shows the effect of unavailable pixels on intra-prediction based upon 8×8 blocks. In both examples, the shaded blocks indicate the blocks that are affected due to the unavailable pixels. As can be observed when comparing these pixels, the larger size transform in FIG. 1(b) is affected to a significantly greater degree than the smaller size transform in FIG. 1(a).